On Optimal Replacement under Semi - Markov Conditions

We study the following model for a system the state of which is continuously observed. The set of possible states is a finite set {0, . . . , L}, where larger values represent increased states of deterioration from the “new condition” represented by state 0, to the “totally inoperative condition” of state L. Whenever the system changes state a decision has to be made as to whether it is renovated or it is left unattended. In this paper we generalize the results of Derman [1] to the case in which the state sojourn times are distributed according to a general state dependent distribution. We also allow a renovation to result in a renovation state l ≥ 0, i.e., the renovated system may not be as good as new. Whenever the system enters a state i < L and the decision to renovate is taken, then a cost c is incurred and its state, immediately, changes to a fixed state l. If the system enters state L then it must be renovated at an increased cost c + A. There is no cost whenever the decision to leave it unattended is taken in a state i < L; in this case the next state will be state j with probability pij and the sojourn time in state i is a random variable with distribution function Fij(·). We provide necessary conditions under which optimal policies are of the “control limit” type. This analysis includes both the continuous time and discrete time cases, and examples are given for both cases. Key–Words: Markovian Decision Processes, Reliability, Nonlinear Optimization.